The present invention generally relates to interconnected heterogeneous data processing systems, and more specifically to reading and writing files by an application on a first system to a disk on a heterogeneous second system.
FIG. 1 is a block diagram illustrating a General Purpose Computer 20 in a data processing system. The General Purpose Computer 20 has a Computer Processor 22, and Memory 24, connected by a Bus 26. Memory 24 is a relatively high speed machine readable medium and includes Volatile Memories such as DRAM, and SRAM, and Non-Volatile Memories such as, ROM, FLASH, EPROM, and EEPROM. Also connected to the Bus are Secondary Storage 30, External Storage 32, output devices such as a monitor 34, input devices such as a keyboard 36 (with mouse 37), and printers 38. Secondary Storage 30 includes machine-readable media such as hard disk drives (or DASD) and disk sub-systems. External Storage 32 includes machine-readable media such as floppy disks, removable hard drives, magnetic tapes, CD-ROM, and even other computers, possibly connected via a communications line 28. The distinction drawn here between Secondary Storage 30 and External Storage 32 is primarily for convenience in describing the invention. As such, it should be appreciated that there is substantial functional overlap between these elements. Computer software such as data base management software, operating systems, and user programs can be stored in a Computer Software Storage Medium, such as memory 24, Secondary Storage 30, and External Storage 32. Executable versions of computer software 33, can be read from a Non-Volatile Storage Medium such as External Storage 32, Secondary Storage 30, and Non-Volatile Memory and loaded for execution directly into Volatile Memory, executed directly out of Non-Volatile Memory, or stored on the Secondary Storage 30 prior to loading into Volatile Memory for execution.
FIG. 2 is a block diagram illustrating file reading and writing across heterogeneous systems, in accordance with the Prior Art. In a first computer system 110, an application 120 writes records to a file 114. When the application 120 completes writing to the file 114, the file 114 is closed. Then, a utility, such as FTP, is utilized to transfer the file 114 to a second computer system 112, where a corresponding utility 124 writes the file 116 on disk on that second computer system 112. A second application 126 can then read and process the second file 116. Any necessary translations between the two heterogeneous computer systems is performed by the two utility programs 122, 124.
In the preferred embodiment of this invention, the first computer system 110 is a GCOS(copyright) 8 mainframe system that operates utilizing 36-bit words with either 4 9-bit or 6 6-bit characters per word. The preferred second computer system 112 is a UNIX system utilizing 8-bit bytes. The preferred UNIX variant is IBM""s AIX. One application that is commonly utilized here is the dumping of a database on the GCOS 8 system 110 to a xe2x80x9cflatxe2x80x9d file 114. The xe2x80x9cflatxe2x80x9d file is then moved as bulk data to a Teradata system 112 from NCR, where the xe2x80x9cflatxe2x80x9d file 114 is loaded into a second database utilizing a xe2x80x9cFastLoadxe2x80x9d program 126 from NCR.
There are a number of problems with this implementation. Most notably, it is necessary to write the data twice, once on each system, and read it twice, again, once on each system. In large systems, this overhead can be substantial.
FIG. 3 is a block diagram illustrating writing of a file 116 on a second computer system 112 by an application 130 executing on a first computer system 110. The file 116 can then be read and processed by an application 136 on the second computer system 112.
This functionality is available in some homogeneous computer systems. For example, the Solaris operating system sold by Sun provides a Remote File System functionality that allows an application on a first computer system 110 to write files 116 on a second computer system 112. Microsoft Windows (various levels) also supports similar functionality.
However, this functionality has been limited in the prior art to homogeneous computer systems such as Solaris or Windows. It has not been available between heterogeneous computer systems. There are a number of reasons for this. One reason that this functionality has been limited in prior art systems to homogeneous computer systems is that in such cases, there is no requirement to perform any translation between systems, such as between 9 and 8 bit bytes as required in the preferred embodiment of this invention.
FIG. 4 is a block diagram illustrating transferring data directly between an application 130 on a first computer system 110 to a second application 136 on a second computer system 112. This is currently available between applications on heterogeneous computer systems as message passing. One example of a message passing mechanism between heterogeneous computer systems is the FlowBus product sold by the assignee of this invention. However, in the prior art, this is typically fairly slow and expensive due to the requirement to acknowledge messages. It would thus be advantageous to provide this functionality between heterogeneous computer systems utilizing more efficient protocols. In particular, it would be advantageous to provide this functionality for bulk data transfers.
Another problem encountered when utilizing heterogeneous computer systems is that of synchronizing jobs executing on the two computer systems 110, 112. Many variants of UNIX provide the capability to start jobs or tasks on other UNIX systems, to wait for results from the execution of those jobs or tasks, and to receive and act upon those results. However, this capability has not been available in the prior art between heterogeneous computer systems. Some of the problems that have prevented this in the prior art are different formats of data on the two systems, different methods of starting jobs or tasks, and different methods of returning job or task status information. It would thus be advantageous to be able to execute jobs or tasks on a second computer system 112 started from a first heterogeneous computer system 110, which then receives the results of that execution when the jobs or tasks complete.
Another problem encountered when utilizing heterogeneous computer systems is that of checkpointing and restarting jobs or tasks operating on. Again, this feature has been present to some extent when operating across multiple homogeneous computer systems, but not across multiple heterogeneous computer systems. Part of the reason for this problem is that each computer architecture involved utilizes its own unique methods of checkpointing and restarting jobs or tasks. It would thus be advantageous to be able to order checkpointing on a second computer system 112 from a first heterogeneous computer system 110, and then later optionally restarting the checkpointed job or task on that second computer system 112.